Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis.

BACKGROUND: The adoption of predictive algorithms in health care comes with the potential for algorithmic bias, which could exacerbate existing disparities. Fairness metrics have been proposed to measure algorithmic bias, but their application to real-world tasks is limited. OBJECTIVE: This study aims to evaluate the algorithmic bias associated with the application of common 30-day hospital readmission models and assess the usefulness and interpretability of selected fairness metrics. METHODS: We used 10.6 million adult inpatient discharges from Maryland and Florida from 2016 to 2019 in this retrospective study. Models predicting 30-day hospital readmissions were evaluated: LACE Index, modified HOSPITAL score, and modified Centers for Medicare & Medicaid Services (CMS) readmission measure, which were applied as-is (using existing coefficients) and retrained (recalibrated with 50% of the data). Predictive performances and bias measures were evaluated for all, between Black and White populations, and between low- and other-income groups. Bias measures included the parity of false negative rate (FNR), false positive rate (FPR), 0-1 loss, and generalized entropy index. Racial bias represented by FNR and FPR differences was stratified to explore shifts in algorithmic bias in different populations. RESULTS: The retrained CMS model demonstrated the best predictive performance (area under the curve: 0.74 in Maryland and 0.68-0.70 in Florida), and the modified HOSPITAL score demonstrated the best calibration (Brier score: 0.16-0.19 in Maryland and 0.19-0.21 in Florida). Calibration was better in White (compared to Black) populations and other-income (compared to low-income) groups, and the area under the curve was higher or similar in the Black (compared to White) populations. The retrained CMS and modified HOSPITAL score had the lowest racial and income bias in Maryland. In Florida, both of these models overall had the lowest income bias and the modified HOSPITAL score showed the lowest racial bias. In both states, the White and higher-income populations showed a higher FNR, while the Black and low-income populations resulted in a higher FPR and a higher 0-1 loss. When stratified by hospital and population composition, these models demonstrated heterogeneous algorithmic bias in different contexts and populations. CONCLUSIONS: Caution must be taken when interpreting fairness measures' face value. A higher FNR or FPR could potentially reflect missed opportunities or wasted resources, but these measures could also reflect health care use patterns and gaps in care. Simply relying on the statistical notions of bias could obscure or underplay the causes of health disparity. The imperfect health data, analytic frameworks, and the underlying health systems must be carefully considered. Fairness measures can serve as a useful routine assessment to detect disparate model performances but are insufficient to inform mechanisms or policy changes. However, such an assessment is an important first step toward data-driven improvement to address existing health disparities.

Ownership and Interoperability Challenges of Alzheimer Monoclonal Antibody Registries.

This Viewpoint addresses the challenges that the Centers for Medicare and Medicaid Services faces to collect real-world data on the effectiveness and safety of lecanemab from external registries to achieve its coverage with evidence development objectives.

Latent Class Analysis of Social Needs in Medicaid Population and Its Impact on Risk Adjustment Models.

BACKGROUND: A growing number of US states are implementing programs to address the social needs (SNs) of their Medicaid populations through managed care contracts. Incorporating SN might also improve risk adjustment methods used to reimburse Medicaid providers. OBJECTIVES: Identify classes of SN present within the Medicaid population and evaluate the performance improvement in risk adjustment models of health care utilization and cost after incorporating SN classes. RESEARCH DESIGN: A secondary analysis of Medicaid patients during the years 2018 and 2019. Latent class analysis (LCA) was used to identify SN classes. To evaluate the impact of SN classes on measures of hospitalization, emergency (ED) visits, and costs, logistic and linear regression modeling for concurrent and prospective years was used. Model performance was assessed before and after incorporating these SN classes to base models controlling for demographics and comorbidities. SUBJECTS: 262,325 Medicaid managed care program patients associated with a large urban academic medical center. RESULTS: 7.8% of the study population had at least one SN, with the most prevalent being related to safety (3.9%). Four classes of SN were determined to be optimal based on LCA, including stress-related needs, safety-related needs, access to health care-related needs, and socioeconomic status-related needs. The addition of SN classes improved the performance of concurrent base models' AUC (0.61 vs. 0.58 for predicting ED visits and 0.61 vs. 0.58 for projecting hospitalizations). CONCLUSIONS: Incorporating SN clusters significantly improved risk adjustment models of health care utilization and costs in the study population. Further investigation into the predictive value of SN for costs and utilization in different Medicaid populations is merited.

Identifying Predictors of Nursing Home Admission by Using Electronic Health Records and Administrative Data: Scoping Review.

Background: Among older adults, nursing home admissions (NHAs) are considered a significant adverse outcome and have been extensively studied. Although the volume and significance of electronic data sources are expanding, it is unclear what predictors of NHA have been systematically identified in the literature via electronic health records (EHRs) and administrative data. Objective: This study synthesizes findings of recent literature on identifying predictors of NHA that are collected from administrative data or EHRs. Methods: The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines were used for study selection. The PubMed and CINAHL databases were used to retrieve the studies. Articles published between January 1, 2012, and March 31, 2023, were included. Results: A total of 34 papers were selected for final inclusion in this review. In addition to NHA, all-cause mortality, hospitalization, and rehospitalization were frequently used as outcome measures. The most frequently used models for predicting NHAs were Cox proportional hazards models (studies: n=12, 35%), logistic regression models (studies: n=9, 26%), and a combination of both (studies: n=6, 18%). Several predictors were used in the NHA prediction models, which were further categorized into sociodemographic, caregiver support, health status, health use, and social service use factors. Only 5 (15%) studies used a validated frailty measure in their NHA prediction models. Conclusions: NHA prediction tools based on EHRs or administrative data may assist clinicians, patients, and policy makers in making informed decisions and allocating public health resources. More research is needed to assess the value of various predictors and data sources in predicting NHAs and validating NHA prediction models externally.

A risk identification model for detection of patients at risk of antidepressant discontinuation.

Purpose: Between 30 and 68% of patients prematurely discontinue their antidepressant treatment, posing significant risks to patient safety and healthcare outcomes. Online healthcare forums have the potential to offer a rich and unique source of data, revealing dimensions of antidepressant discontinuation that may not be captured by conventional data sources. Methods: We analyzed 891 patient narratives from the online healthcare forum, "askapatient.com," utilizing content analysis to create PsyRisk-a corpus highlighting the risk factors associated with antidepressant discontinuation. Leveraging PsyRisk, alongside PsyTAR [a publicly available corpus of adverse drug reactions (ADRs) related to antidepressants], we developed a machine learning-driven algorithm for proactive identification of patients at risk of abrupt antidepressant discontinuation. Results: From the analyzed 891 patients, 232 reported antidepressant discontinuation. Among these patients, 92% experienced ADRs, and 72% found these reactions distressful, negatively affecting their daily activities. Approximately 26% of patients perceived the antidepressants as ineffective. Most reported ADRs were physiological (61%, 411/673), followed by cognitive (30%, 197/673), and psychological (28%, 188/673) ADRs. In our study, we employed a nested cross-validation strategy with an outer 5-fold cross-validation for model selection, and an inner 5-fold cross-validation for hyperparameter tuning. The performance of our risk identification algorithm, as assessed through this robust validation technique, yielded an AUC-ROC of 90.77 and an F1-score of 83.33. The most significant contributors to abrupt discontinuation were high perceived distress from ADRs and perceived ineffectiveness of the antidepressants. Conclusion: The risk factors identified and the risk identification algorithm developed in this study have substantial potential for clinical application. They could assist healthcare professionals in identifying and managing patients with depression who are at risk of prematurely discontinuing their antidepressant treatment.

Assessing Patient and Community-Level Social Factors; The Synergistic Effect of Social Needs and Social Determinants of Health on Healthcare Utilization at a Multilevel Academic Healthcare System.

We investigated the role of both individual-level social needs and community-level social determinants of health (SDOH) in explaining emergency department (ED) utilization rates. We also assessed the potential synergies between the two levels of analysis and their combined effect on patterns of ED visits. We extracted electronic health record (EHR) data between July 2016 and June 2020 for 1,308,598 unique Maryland residents who received care at Johns Hopkins Health System, of which 28,937 (2.2%) patients had at least one documented social need. There was a negative correlation between median household income in a neighborhood with having a social need such as financial resource strain, food insecurity, and residential instability (correlation coefficient: -0.05, -0.01, and - 0.06, p = 0, respectively). In a multilevel model with random effects after adjusting for other factors, living in a more disadvantaged neighborhood was found to be significantly associated with ED utilization statewide and within Baltimore City (OR: 1.005, 95% CI: 1.003-1.007 and 1.020, 95% CI: 1.017-1.022, respectively). However, individual-level social needs appeared to enhance the statewide effect of living in a more disadvantaged neighborhood with the OR for the interaction term between social needs and SDOH being larger, and more positive, than SDOH alone (OR: 1.012, 95% CI: 1.011-1.014). No such moderation was found in Baltimore City. To our knowledge, this study is one of the first attempts by a major academic healthcare system to assess the combined impact of patient-level social needs in association with community-level SDOH on healthcare utilization and can serve as a baseline for future studies using EHR data linked to population-level data to assess such synergistic association.

Prevalence and Correlates of Opioid-Involved Suicides in Maryland.

OBJECTIVE: Involvement of opioids in suicides has doubled during the past two decades, worsening a major public health concern. This study examined the characteristics of opioid-involved suicides. METHODS: The sample of decedents (N = 12,038) in Maryland between 2006 and 2020 was used to compare the characteristics of opioid-involved suicides (n = 947) with suicides not involving opioids (n = 6,896) and accidental opioid deaths (n = 4,125). Direct comparisons were then made between opioid-involved suicides with and without the additional presence of non-opioid substances. RESULTS: Opioid-involved suicides were significantly more likely than suicides not involving opioids to occur among those aged 18-64 years, non-Hispanic Whites, and unemployed or disabled individuals. Opioid-involved suicides were more likely than accidental opioid deaths to occur among females, those aged <18 years, non-Hispanic Whites, and employed individuals. Of all suicides involved opioids, 45% involved other non-opioid substances. Polysubstance opioid suicides were significantly more likely than suicides involving opioids only to occur among non-Hispanic Whites. CONCLUSIONS: Significant differences were observed in the demographic groups most at risk for opioid-involved suicide than other suicide or accidental opioid death. Among opioid-involved suicides, polysubstance involvement also represents a distinct group. These findings may enhance the targeting of prevention efforts.HIGHLIGHTSOpioid-involved suicides differ from other suicides or accidental opioid deaths.In opioid-involved suicides, polysubstance involvement represents a distinct group.These findings may enhance the targeting of prevention efforts.

Health Disparities in Lapses in Diabetic Retinopathy Care.

Objective: To develop a novel methodology to identify lapses in diabetic retinopathy care in electronic health records (EHRs) and evaluate health disparities by race and ethnicity. Design: Retrospective cohort study. Subjects: Adult patients with diabetes mellitus who were evaluated at the Wilmer Eye Institute from January 1, 2013 to April 2, 2022. Methods: The methodology to identify lapses in care first identified diabetic retinopathy screening or treatment visits and then compared the providers' recommended follow-up timeframe with the patient's actual time to next encounter. The association of race and ethnicity with odds of lapses in care was evaluated using a mixed-effects logistic regression model controlling for age, sex, insurance, severity of diabetic retinopathy, presence of other retinal disorders, and glaucoma. Main Outcome Measures: Lapses in diabetic retinopathy care. Results: The methodology to identify diabetic retinopathy-related visits had a 95.0% (95% confidence interval, 93.0-96.6) sensitivity and 98.8% (98.1-99.3) specificity as compared with a gold standard grader. The methodology resulted in a 97.3% (96.2-98.4) sensitivity and 98.1% (97.3-98.9) specificity for detecting a follow-up recommendation, with an average error of -0.05 (-0.31 to 0.21) weeks in extracting the precise timeframe. A total of 39 561 patients with 91 104 office visits were included in the analysis. The average age was 61.4 years. More than 3 (77.6%) in 4 patients had a lapse in care. In multivariable analysis, non-Hispanic Black patients had 1.24 (1.19-1.30) odds and Hispanic patients had 1.26 (1.13-1.40) odds of ever having a lapse in care compared with non-Hispanic White patients (P < 0.001, respectively). Conclusions: We have developed a reliable methodology for identifying lapses in diabetic retinopathy care that is tailored to a provider's recommended follow-up. Using this approach, we find that 3 in 4 patients experience a lapse in diabetic retinopathy care and that these rates are higher among non-Hispanic Black and Hispanic patients. Deploying this methodology in the EHR is one potential means by which to identify and mitigate lapses in critical ophthalmic care in patients with diabetes. Financial Disclosures: Proprietary or commercial disclosure may be found after the references.

Trends and Patterns of Social History Data Collection Within an Electronic Health Record.

There is increased acceptance that social and behavioral determinants of health (SBDH) impact health outcomes, but electronic health records (EHRs) are not always set up to capture the full range of SBDH variables in a systematic manner. The purpose of this study was to explore rates and trends of social history (SH) data collection-1 element of SBDH-in a structured portion of an EHR within a large academic integrated delivery system. EHR data for individuals with at least 1 visit in 2017 were included in this study. Completeness rates were calculated for how often SBDH variable was assessed and documented. Logistic regressions identified factors associated with assessment rates for each variable. A total of 44,166 study patients had at least 1 SH variable present. Tobacco use and alcohol use were the most frequently captured SH variables. Black individuals were more likely to have their alcohol use assessed (odds ratio [OR] 1.21) compared with White individuals, whereas White individuals were more likely to have their "smokeless tobacco use" assessed (OR 0.92). There were also differences between insurance types. Drug use was more likely to be assessed in the Medicaid population for individuals who were single (OR 0.95) compared with the commercial population (OR 1.05). SH variable assessment is inconsistent, which makes use of EHR data difficult to gain better understanding of the impact of SBDH on health outcomes. Standards and guidelines on how and why to collect SBDH information within the EHR are needed.

Measuring the geographic disparity of comorbidity in commercially insured individuals compared to the distribution of physicians in South Africa.

BACKGROUND: Measuring and addressing the disparity between access to healthcare resources and underlying health needs of populations is a prominent focus in health policy development. More recently, the fair distribution of healthcare resources among population subgroups have become an important indication of health inequities. Single disease outcomes are commonly used for healthcare resource allocations; however, leveraging population-level comorbidity measures for health disparity research has been limited. This study compares the geographical distribution of comorbidity and associated healthcare utilization among commercially insured individuals in South Africa (SA) relative to the distribution of physicians. METHODS: A retrospective, cross-sectional analysis was performed comparing the geographical distribution of comorbidity and physicians for 2.6 million commercially insured individuals over 2016-2017, stratified by geographical districts and population groups in SA. We applied the Johns Hopkins ACG® System across the claims data of a large health plan administrator to measure a comorbidity risk score for each individual. By aggregating individual scores, we determined the average healthcare resource need of individuals per district, known as the comorbidity index (CMI), to describe the disease burden per district. Linear regression models were constructed to test the relationship between CMI, age, gender, population group, and population density against physician density. RESULTS: Our results showed a tendency for physicians to practice in geographic areas with more insurance enrollees and not necessarily where disease burden may be highest. This was confirmed by a negative relationship between physician density and CMI for the overall population and for three of the four major population groups. Among the population groups, the Black African population had, on average, access to fewer physicians per capita than other population groups, before and after adjusting for confounding factors. CONCLUSION: CMI is a novel measure for healthcare disparities research that considers both acute and chronic conditions contributing to current and future healthcare costs. Our study linked and compared the population-level geographical distribution of CMI to the distribution of physicians using routinely collected data. Our results could provide vital information towards the more equitable distribution of healthcare providers across population groups in SA, and to meet the healthcare needs of disadvantaged communities.

Assessing the Racial and Socioeconomic Disparities in Postpartum Depression Using Population-Level Hospital Discharge Data: Longitudinal Retrospective Study.

BACKGROUND: In the United States, >3.6 million deliveries occur annually. Among them, up to 20% (approximately 700,000) of women experience postpartum depression (PPD) according to the Centers for Disease Control and Prevention. Absence of accurate reporting and diagnosis has made phenotyping of patients with PPD difficult. Existing literature has shown that factors such as race, socioeconomic status, and history of substance abuse are associated with the differential risks of PPD. However, limited research has considered differential temporal associations with the outcome. OBJECTIVE: This study aimed to estimate the disparities in the risk of PPD and time to diagnosis for patients of different racial and socioeconomic backgrounds. METHODS: This is a longitudinal retrospective study using the statewide hospital discharge data from Maryland. We identified 160,066 individuals who had a hospital delivery from 2017 to 2019. We applied logistic regression and Cox regression to study the risk of PPD across racial and socioeconomic strata. Multinomial regression was used to estimate the risk of PPD at different postpartum stages. RESULTS: The cumulative incidence of PPD diagnosis was highest for White patients (8779/65,028, 13.5%) and lowest for Asian and Pacific Islander patients (248/10,760, 2.3%). Compared with White patients, PPD diagnosis was less likely to occur for Black patients (odds ratio [OR] 0.31, 95% CI 0.30-0.33), Asian or Pacific Islander patients (OR 0.17, 95% CI 0.15-0.19), and Hispanic patients (OR 0.21, 95% CI 0.19-0.22). Similar findings were observed from the Cox regression analysis. Multinomial regression showed that compared with White patients, Black patients (relative risk 2.12, 95% CI 1.73-2.60) and Asian and Pacific Islander patients (relative risk 2.48, 95% CI 1.46-4.21) were more likely to be diagnosed with PPD after 8 weeks of delivery. CONCLUSIONS: Compared with White patients, PPD diagnosis is less likely to occur in individuals of other races. We found disparate timing in PPD diagnosis across different racial groups and socioeconomic backgrounds. Our findings serve to enhance intervention strategies and policies for phenotyping patients at the highest risk of PPD and to highlight needs in data quality to support future work on racial disparities in PPD.

Assessing the Added Value of Vital Signs Extracted from Electronic Health Records in Healthcare Risk Adjustment Models.

Purpose: Patient vital signs are related to specific health risks and outcomes but are underutilized in the prediction of health-care utilization and cost. To measure the added value of electronic health record (EHR) extracted Body Mass Index (BMI) and blood pressure (BP) values in improving healthcare risk and utilization predictions. Patients and Methods: A sample of 12,820 adult outpatients from the Johns Hopkins Health System (JHHS) were identified between 2016 and 2017, having high data quality and recorded values for BMI and BP. We evaluated the added value of BMI and BP in predicting health-care utilization and cost through a retrospective cohort design. BMI, mean arterial pressure (MAP), systolic and diastolic BPs were summarized as annual aggregated values. Concurrent annual BMI and MAP changes were quantified as the difference between maximum and minimum recorded values. Model performance estimates consisted of repeated 10-fold cross validation, compared to base model point estimates for demographic and diagnostic, coded events: (1) patient age and sex, (2) age, sex, and the Charlson weighted index, (3) age, sex and the Johns Hopkins ACG system's DxPM risk score. Results: Both categorical BMI and BP were progressively indicative of disease comorbidity, but not uniformly related to health-care utilization or cost. Annual change in BMI and MAP improved predictions for most concurrent year outcomes when compared to base models. Conclusion: When a healthcare system lacks relevant diagnostic or risk assessment information for a patient, vital signs may be useful for a simple estimation of disease risk, cost and utilization.

Impact of Social Needs in Electronic Health Records and Claims on Health Care Utilization and Costs Risk-Adjustment Models Within Medicaid Population.

Patients enrolled in Medicaid have significantly higher social needs (SNs) than others. Using claims and electronic health records (EHRs) data, managed care organizations (MCOs) could systemically identify high-risk patients with SNs and develop population health management interventions. Impact of SNs on models predicting health care utilization and costs was assessed. This retrospective study included claims and EHRs data on 39,267 patients younger than age 65 years who were continuously enrolled during 2018-2019 in a Medicaid-managed care plan. SN marker was developed suggesting presence of International Classification of Diseases, 10th revision codes in any of the 5 SN domains. Impact of SN marker was compared across demographic and 2 diagnosis-based (ie, Charlson and Adjusted Clinical Groups risk score) prediction models of emergency department (ED) visit and hospitalizations, and total, medical, and pharmacy costs. After combining data sources, prevalence of documented SN marker increased from 11% and 13% to 18% of the study population across claims, EHRs, and both combined, respectively. SN marker improved predictions of demographic models for all utilization and total costs outcomes (area under the curve [AUC] of ED model increased from 0.57 to 0.61 and R2 of total cost model increased from 10.9 to 12.2). In both diagnosis-based models, adding SN marker marginally improved outcomes prediction (AUC of ED model increased from 0.65 to 0.66). This study demonstrated feasibility of using claims and EHRs data to systematically capture SNs and incorporate in prediction models that could enable MCOs and policy makers to adjust and develop effective population health interventions.

Impact of social determinants of health on improving the LACE index for 30-day unplanned readmission prediction.

Objective: Early and accurate prediction of patients at risk of readmission is key to reducing costs and improving outcomes. LACE is a widely used score to predict 30-day readmissions. We examine whether adding social determinants of health (SDOH) to LACE can improve its predictive performance. Methods: This is a retrospective study that included all inpatient encounters in the state of Maryland in 2019. We constructed predictive models by fitting Logistic Regression (LR) on LACE and different sets of SDOH predictors. We used the area under the curve (AUC) to evaluate discrimination and SHapley Additive exPlanations values to assess feature importance. Results: Our study population included 316 558 patients of whom 35 431 (11.19%) patients were readmitted after 30 days. Readmitted patients had more challenges with individual-level SDOH and were more likely to reside in communities with poor SDOH conditions. Adding a combination of individual and community-level SDOH improved LACE performance from AUC = 0.698 (95% CI [0.695-0.7]; ref) to AUC = 0.708 (95% CI [0.705-0.71]; P < .001). The increase in AUC was highest in black patients (+1.6), patients aged 65 years or older (+1.4), and male patients (+1.4). Discussion: We demonstrated the value of SDOH in improving the LACE index. Further, the additional predictive value of SDOH on readmission risk varies by subpopulations. Vulnerable populations like black patients and the elderly are likely to benefit more from the inclusion of SDOH in readmission prediction. Conclusion: These findings provide potential SDOH factors that health systems and policymakers can target to reduce overall readmissions.

Claims-based pharmacy markers for comprehensive medication management program case identification: Validation against concurrent and prospective healthcare costs and utilization.

BACKGROUND: Three claims-based pharmacy markers (complex, costly and risky medications) were developed to help automatically identify patients for comprehensive medication management. OBJECTIVE: To evaluate the association between newly-developed markers and healthcare outcomes. METHODS: This was a two-year retrospective cohort study using PharMetrics Plus patient-level administrative claims in 2014 and 2015. We included all claims from 1,541,873 individuals with: (1) 24-month medical and pharmacy enrollment in 2014 and 2015, (2) aged between 18 and 63 in 2014, and (3) known gender. Independent/control variables came from 2014 while outcomes came from 2014 (concurrent analysis) and 2015 (prospective analysis). Three pharmacy markers, separately or together, were added to four base models to predict concurrent and prospective healthcare costs (total, medical, and pharmacy) and utilization (having any hospitalization, having any emergency department visit, and having any readmission). We applied linear regression for costs while logistic regression for utilization. Measures of model performances and coefficients were derived from a 5-fold cross-validation repeated 20 times. RESULTS: Individuals with 1+ complex, risky or costly medication markers had higher comorbidity, healthcare costs and utilization than their counterparts. Nine binary risky category markers performed the best among the three types of risky medication markers; the Medication Complexity Score and three-level complex category both outperformed a simpler complex medication indicator. Adding three novel pharmacy markers separately or together into the base models provided the greatest improvement in explaining pharmacy costs, compared with medical (non-medication) costs. These pharmacy markers also added value in explaining healthcare utilization among the simple base models. CONCLUSIONS: Three claims-based pharmacy indicators had positive associations with healthcare outcomes and added value in predicting them. This initial study suggested that these novel markers can be used by pharmacy case management programs to help identify potential high-risk patients most likely to benefit from clinical pharmacist review and other interventions.

Comparing person-level matching algorithms to identify risk across disparate datasets among patients with a controlled substance prescription: retrospective analysis.

Background: The opioid epidemic in the United States has precipitated a need for public health agencies to better understand risk factors associated with fatal overdoses. Matching person-level information stored in public health, medical, and human services datasets can enhance the understanding of opioid overdose risk factors and interventions. Objective: This study compares approximate match versus exact match algorithms to link disparate datasets together for identifying persons at risk from an applied perspective. Methods: This study used statewide prescription drug monitoring program (PDMP), arrest, and mortality data matched at the person-level using an approximate match and 2 exact match algorithms. Impact of matching was assessed by analyzing 3 independent concepts: (1) the prevalence of key risk indicators used by PDMP programs in practice, (2) the prevalence of arrests and fatal opioid overdose, and (3) the performance of a multivariate logistic regression for fatal opioid overdose. The PDMP key risk indicators included (1) multiple provider episodes (MPE), or patients with prescriptions from multiple prescribers and dispensers, (2) high morphine milligram equivalents (MMEs), which represents an opioid's potency relative to morphine, and (3) overlapping opioid and benzodiazepine prescriptions. Results: Prevalence of PDMP-based risk indicators were higher in the approximate match population for MPEs (n = 4893/1 859 445 [0.26%]) and overlapping opioid/benzodiazepines (n = 57 888/1 859 445 [4.71%]), but the exact-basic match population had the highest prevalence of individuals with high MMEs (n = 664/1 910 741 [3.11%]). Prevalence of arrests and deaths were highest for the approximate match population compared with the exact match populations. Model performance was comparable across the 3 matching algorithms (exact-basic validation area under the receiver operating characteristic curve [AUC]: 0.854; approximate validation AUC: 0.847; exact + zip validation AUC: 0.826) but resulted in different cutoff points balancing sensitivity and specificity. Conclusions: Our study illustrates the specific tradeoffs of different matching methods. Further research should be performed to compare matching algorithms and its impact on the prevalence of key risk indicators in an applied setting that can improve understanding of risk within a population.

A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models.

OBJECTIVE: Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model's potential to introduce bias. MATERIALS AND METHODS: Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist. RESULTS: We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern. DISCUSSION: Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed. CONCLUSION: The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

Clinical, social, and policy factors in COVID-19 cases and deaths: methodological considerations for feature selection and modeling in county-level analyses.

BACKGROUND: There is a need to evaluate how the choice of time interval contributes to the lack of consistency of SDoH variables that appear as important to COVID-19 disease burden within an analysis for both case counts and death counts. METHODS: This study identified SDoH variables associated with U.S county-level COVID-19 cumulative case and death incidence for six different periods: the first 30, 60, 90, 120, 150, and 180 days since each county had COVID-19 one case per 10,000 residents. The set of SDoH variables were in the following domains: resource deprivation, access to care/health resources, population characteristics, traveling behavior, vulnerable populations, and health status. A generalized variance inflation factor (GVIF) analysis was used to identify variables with high multicollinearity. For each dependent variable, a separate model was built for each of the time periods. We used a mixed-effect generalized linear modeling of counts normalized per 100,000 population using negative binomial regression. We performed a Kolmogorov-Smirnov goodness of fit test, an outlier test, and a dispersion test for each model. Sensitivity analysis included altering the county start date to the day each county reached 10 COVID-19 cases per 10,000. RESULTS: Ninety-seven percent (3059/3140) of the counties were represented in the final analysis. Six features proved important for both the main and sensitivity analysis: adults-with-college-degree, days-sheltering-in-place-at-start, prior-seven-day-median-time-home, percent-black, percent-foreign-born, over-65-years-of-age, black-white-segregation, and days-since-pandemic-start. These variables belonged to the following categories: COVID-19 related, vulnerable populations, and population characteristics. Our diagnostic results show that across our outcomes, the models of the shorter time periods (30 days, 60 days, and 900 days) have a better fit. CONCLUSION: Our findings demonstrate that the set of SDoH features that are significant for COVID-19 outcomes varies based on the time from the start date of the pandemic and when COVID-19 was present in a county. These results could assist researchers with variable selection and inform decision makers when creating public health policy.